Pandas Profiling Report for Nhabe Radar dataset

Dataset statistics

Number of variables	8
Number of observations	703000
Missing cells	2466634
Missing cells (%)	43.9%
Duplicate rows	55
Duplicate rows (%)	< 0.1%
Total size in memory	42.9 MiB
Average record size in memory	64.0 B

Variable types

Numeric	8

Alerts

Dataset has 55 (< 0.1%) duplicate rows	Duplicates
`reflectivity` is highly overall correlated with `total_power`	High correlation
`total_power` is highly overall correlated with `reflectivity`	High correlation
`reflectivity` has 593838 (84.5%) missing values	Missing
`total_power` has 562225 (80.0%) missing values	Missing
`velocity` has 651972 (92.7%) missing values	Missing
`spectrum_width` has 658599 (93.7%) missing values	Missing

Reproduction

Analysis started	2024-04-22 10:26:24.190508
Analysis finished	2024-04-22 10:26:32.197596
Duration	8.01 seconds
Software version	ydata-profiling vv4.7.0
Download configuration	config.json

longitude
Real number (ℝ)

Distinct	636062
Distinct (%)	90.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	10.686026

Minimum	7.9688983
Maximum	13.349995
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.4 MiB

Quantile statistics

Minimum	7.9688983
5-th percentile	8.774408
Q1	10.00852
median	10.673027
Q3	11.376181
95-th percentile	12.586302
Maximum	13.349995
Range	5.3810967
Interquartile range (IQR)	1.3676605

Descriptive statistics

Standard deviation	1.0897991
Coefficient of variation (CV)	0.10198358
Kurtosis	-0.26925643
Mean	10.686026
Median Absolute Deviation (MAD)	0.683974
Skewness	-0.014574397
Sum	7512276.3
Variance	1.1876622
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10.65961	1407	0.2%
10.65966	7	< 0.1%
10.659811	7	< 0.1%
10.659615	7	< 0.1%
10.659808	6	< 0.1%
10.659659	6	< 0.1%
10.659332	6	< 0.1%
10.659616	6	< 0.1%
10.708096	6	< 0.1%
10.659797	6	< 0.1%
Other values (636052)	701536	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
7.9688983	1	< 0.1%
7.969086	1	< 0.1%
7.9692225	1	< 0.1%
7.9704094	1	< 0.1%
7.9705014	1	< 0.1%
7.9718328	1	< 0.1%
7.97201	1	< 0.1%
7.972188	1	< 0.1%
7.9725595	1	< 0.1%
7.9731936	1	< 0.1%

Value	Count	Frequency (%)
13.349995	1	< 0.1%
13.349702	1	< 0.1%
13.349603	1	< 0.1%
13.348747	1	< 0.1%
13.348638	1	< 0.1%
13.347436	1	< 0.1%
13.347107	1	< 0.1%
13.346885	1	< 0.1%
13.346762	1	< 0.1%
13.346106	1	< 0.1%

latitude
Real number (ℝ)

Distinct	390890
Distinct (%)	55.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	106.74303

Minimum	103.99098
Maximum	109.46626
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.4 MiB

Quantile statistics

Minimum	103.99098
5-th percentile	104.76653
Q1	106.03384
median	106.74245
Q3	107.45951
95-th percentile	108.69125
Maximum	109.46626
Range	5.475275
Interquartile range (IQR)	1.4256725

Descriptive statistics

Standard deviation	1.1243545
Coefficient of variation (CV)	0.010533283
Kurtosis	-0.31753025
Mean	106.74303
Median Absolute Deviation (MAD)	0.71285
Skewness	-0.028459393
Sum	75040350
Variance	1.2641731
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
106.728325	1410	0.2%
106.728035	25	< 0.1%
106.72947	24	< 0.1%
106.72823	23	< 0.1%
106.7326	23	< 0.1%
106.72993	22	< 0.1%
106.72797	20	< 0.1%
106.73175	19	< 0.1%
106.73312	18	< 0.1%
106.728165	18	< 0.1%
Other values (390880)	701398	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
103.99098	1	< 0.1%
103.991005	1	< 0.1%
103.991875	1	< 0.1%
103.99193	1	< 0.1%
103.99335	1	< 0.1%
103.99346	1	< 0.1%
103.99348	1	< 0.1%
103.993614	1	< 0.1%
103.99411	1	< 0.1%
103.99427	1	< 0.1%

Value	Count	Frequency (%)
109.466255	1	< 0.1%
109.46624	1	< 0.1%
109.46547	1	< 0.1%
109.465324	1	< 0.1%
109.46379	1	< 0.1%
109.46374	1	< 0.1%
109.463356	1	< 0.1%
109.463326	1	< 0.1%
109.4626	1	< 0.1%
109.46243	1	< 0.1%

altitude
Real number (ℝ)

Distinct	23421
Distinct (%)	3.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	7129.0934

Minimum	10
Maximum	25398
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.4 MiB

Quantile statistics

Minimum	10
5-th percentile	365.95
Q1	2389
median	5770
Q3	10784
95-th percentile	18470
Maximum	25398
Range	25388
Interquartile range (IQR)	8395

Descriptive statistics

Standard deviation	5742.2345
Coefficient of variation (CV)	0.80546491
Kurtosis	-0.0088721892
Mean	7129.0934
Median Absolute Deviation (MAD)	3892
Skewness	0.86464998
Sum	5.0117526 × 10⁹
Variance	32973257
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10	1406	0.2%
26	382	0.1%
59	306	< 0.1%
35	289	< 0.1%
85	272	< 0.1%
92	250	< 0.1%
43	240	< 0.1%
60	230	< 0.1%
86	225	< 0.1%
48	211	< 0.1%
Other values (23411)	699189	99.5%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
10	1406	0.2%
13	54	< 0.1%
14	129	< 0.1%
15	103	< 0.1%
16	19	< 0.1%
17	37	< 0.1%
18	78	< 0.1%
19	64	< 0.1%
20	62	< 0.1%
21	81	< 0.1%

Value	Count	Frequency (%)
25398	1	< 0.1%
25336	1	< 0.1%
25326	1	< 0.1%
25298	2	< 0.1%
25275	1	< 0.1%
25265	1	< 0.1%
25241	1	< 0.1%
25237	2	< 0.1%
25214	1	< 0.1%
25204	1	< 0.1%

reflectivity
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	5532
Distinct (%)	5.1%
Missing	593838
Missing (%)	84.5%
Infinite	0
Infinite (%)	0.0%
Mean	4.2562447

Minimum	-23.93
Maximum	59.27
Zeros	47
Zeros (%)	< 0.1%
Negative	38566
Negative (%)	5.5%
Memory size	5.4 MiB

Quantile statistics

Minimum	-23.93
5-th percentile	-13.99
Q1	-4.16
median	4.58
Q3	11.82
95-th percentile	23.2295
Maximum	59.27
Range	83.2
Interquartile range (IQR)	15.98

Descriptive statistics

Standard deviation	11.177856
Coefficient of variation (CV)	2.6262251
Kurtosis	-0.51854467
Mean	4.2562447
Median Absolute Deviation (MAD)	7.92
Skewness	0.096668684
Sum	464620.18
Variance	124.94447
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
7.46	60	< 0.1%
7.41	57	< 0.1%
6.71	57	< 0.1%
8.05	56	< 0.1%
7.01	56	< 0.1%
3.98	56	< 0.1%
8.61	56	< 0.1%
8.74	55	< 0.1%
5.19	54	< 0.1%
6.44	54	< 0.1%
Other values (5522)	108601	15.4%
(Missing)	593838	84.5%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-23.93	1	< 0.1%
-23.63	1	< 0.1%
-23.18	1	< 0.1%
-22.84	1	< 0.1%
-22.56	1	< 0.1%
-22.45	1	< 0.1%
-22.41	1	< 0.1%
-22.3	1	< 0.1%
-22.26	1	< 0.1%
-22.17	1	< 0.1%

Value	Count	Frequency (%)
59.27	1	< 0.1%
56.65	1	< 0.1%
51.95	1	< 0.1%
49.37	1	< 0.1%
49.09	1	< 0.1%
48.92	1	< 0.1%
47.84	1	< 0.1%
47.8	1	< 0.1%
47.68	1	< 0.1%
47.42	1	< 0.1%

total_power
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	7201
Distinct (%)	5.1%
Missing	562225
Missing (%)	80.0%
Infinite	0
Infinite (%)	0.0%
Mean	8.8439624

Minimum	-19.03
Maximum	75.66
Zeros	53
Zeros (%)	< 0.1%
Negative	36050
Negative (%)	5.1%
Memory size	5.4 MiB

Quantile statistics

Minimum	-19.03
5-th percentile	-9.233
Q1	-0.23
median	7.53
Q3	16.56
95-th percentile	30.74
Maximum	75.66
Range	94.69
Interquartile range (IQR)	16.79

Descriptive statistics

Standard deviation	12.6009
Coefficient of variation (CV)	1.4248025
Kurtosis	0.76134825
Mean	8.8439624
Median Absolute Deviation (MAD)	8.32
Skewness	0.68897027
Sum	1245008.8
Variance	158.78268
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
23.33	72	< 0.1%
4.63	70	< 0.1%
23.35	69	< 0.1%
7.66	68	< 0.1%
6.2	68	< 0.1%
6.53	67	< 0.1%
7.26	67	< 0.1%
5.95	66	< 0.1%
4.59	66	< 0.1%
5.37	66	< 0.1%
Other values (7191)	140096	19.9%
(Missing)	562225	80.0%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-19.03	1	< 0.1%
-19.02	1	< 0.1%
-18.97	1	< 0.1%
-18.76	1	< 0.1%
-18.66	1	< 0.1%
-18.57	1	< 0.1%
-18.53	1	< 0.1%
-18.49	1	< 0.1%
-18.39	1	< 0.1%
-18.38	1	< 0.1%

Value	Count	Frequency (%)
75.66	1	< 0.1%
74.28	1	< 0.1%
70.81	1	< 0.1%
70.67	1	< 0.1%
70.22	1	< 0.1%
69.81	1	< 0.1%
69.42	1	< 0.1%
69.13	1	< 0.1%
68.7	1	< 0.1%
68.59	1	< 0.1%

velocity
Real number (ℝ)

MISSING

Distinct	805
Distinct (%)	1.6%
Missing	651972
Missing (%)	92.7%
Infinite	0
Infinite (%)	0.0%
Mean	-0.13457514

Minimum	-4.02
Maximum	4.02
Zeros	42
Zeros (%)	< 0.1%
Negative	26646
Negative (%)	3.8%
Memory size	5.4 MiB

Quantile statistics

Minimum	-4.02
5-th percentile	-3.69
Q1	-2.39
median	-0.26
Q3	2.04
95-th percentile	3.68
Maximum	4.02
Range	8.04
Interquartile range (IQR)	4.43

Descriptive statistics

Standard deviation	2.439908
Coefficient of variation (CV)	-18.130452
Kurtosis	-1.3011469
Mean	-0.13457514
Median Absolute Deviation (MAD)	2.21
Skewness	0.074616526
Sum	-6867.1
Variance	5.953151
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-3.88	115	< 0.1%
-3.6	102	< 0.1%
-3.67	98	< 0.1%
-3.39	97	< 0.1%
-3.51	96	< 0.1%
-3.28	96	< 0.1%
-3.87	95	< 0.1%
3.72	95	< 0.1%
-3.61	95	< 0.1%
-3.47	95	< 0.1%
Other values (795)	50044	7.1%
(Missing)	651972	92.7%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-4.02	33	< 0.1%
-4.01	93	< 0.1%
-4	75	< 0.1%
-3.99	68	< 0.1%
-3.98	72	< 0.1%
-3.97	78	< 0.1%
-3.96	60	< 0.1%
-3.95	73	< 0.1%
-3.94	72	< 0.1%
-3.93	70	< 0.1%

Value	Count	Frequency (%)
4.02	34	< 0.1%
4.01	77	< 0.1%
4	80	< 0.1%
3.99	64	< 0.1%
3.98	82	< 0.1%
3.97	80	< 0.1%
3.96	82	< 0.1%
3.95	89	< 0.1%
3.94	85	< 0.1%
3.93	71	< 0.1%

spectrum_width
Real number (ℝ)

MISSING

Distinct	219
Distinct (%)	0.5%
Missing	658599
Missing (%)	93.7%
Infinite	0
Infinite (%)	0.0%
Mean	0.73154997

Minimum	0.01
Maximum	2.35
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.4 MiB

Quantile statistics

Minimum	0.01
5-th percentile	0.28
Q1	0.53
median	0.71
Q3	0.91
95-th percentile	1.27
Maximum	2.35
Range	2.34
Interquartile range (IQR)	0.38

Descriptive statistics

Standard deviation	0.30548474
Coefficient of variation (CV)	0.41758561
Kurtosis	0.7527892
Mean	0.73154997
Median Absolute Deviation (MAD)	0.19
Skewness	0.47709307
Sum	32481.55
Variance	0.093320926
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.65	678	0.1%
0.61	676	0.1%
0.62	675	0.1%
0.6	649	0.1%
0.69	649	0.1%
0.68	646	0.1%
0.64	635	0.1%
0.71	634	0.1%
0.58	631	0.1%
0.72	626	0.1%
Other values (209)	37902	5.4%
(Missing)	658599	93.7%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0.01	611	0.1%
0.02	5	< 0.1%
0.03	6	< 0.1%
0.04	6	< 0.1%
0.05	17	< 0.1%
0.06	16	< 0.1%
0.07	22	< 0.1%
0.08	20	< 0.1%
0.09	28	< 0.1%
0.1	30	< 0.1%

Value	Count	Frequency (%)
2.35	1	< 0.1%
2.33	1	< 0.1%
2.31	1	< 0.1%
2.26	1	< 0.1%
2.23	1	< 0.1%
2.22	2	< 0.1%
2.21	1	< 0.1%
2.2	1	< 0.1%
2.16	1	< 0.1%
2.14	1	< 0.1%

time
Real number (ℝ)

Distinct	160
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	87.385491

Minimum	0
Maximum	172
Zeros	4000
Zeros (%)	0.6%
Negative	0
Negative (%)	0.0%
Memory size	5.4 MiB

Quantile statistics

Minimum	0
5-th percentile	8
Q1	48
median	90
Q3	133
95-th percentile	164
Maximum	172
Range	172
Interquartile range (IQR)	85

Descriptive statistics

Standard deviation	50.443468
Coefficient of variation (CV)	0.57725222
Kurtosis	-1.1899074
Mean	87.385491
Median Absolute Deviation (MAD)	42
Skewness	-0.079872632
Sum	61432000
Variance	2544.5435
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
83	5500	0.8%
96	5500	0.8%
116	5500	0.8%
147	5500	0.8%
135	5500	0.8%
167	5500	0.8%
71	5500	0.8%
54	5500	0.8%
155	5500	0.8%
59	5000	0.7%
Other values (150)	648500	92.2%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0	4000	0.6%
1	4500	0.6%
2	5000	0.7%
3	4000	0.6%
4	4500	0.6%
5	4500	0.6%
6	5000	0.7%
7	3500	0.5%
8	5000	0.7%
9	4500	0.6%

Value	Count	Frequency (%)
172	500	0.1%
171	4000	0.6%
170	4000	0.6%
169	4500	0.6%
168	4000	0.6%
167	5500	0.8%
166	4500	0.6%
165	4000	0.6%
164	5000	0.7%
163	4500	0.6%

Auto

Heatmap
Table

	altitude	latitude	longitude	reflectivity	spectrum_width	time	total_power	velocity
altitude	1.000	-0.008	-0.005	0.363	0.048	0.456	-0.160	-0.041
latitude	-0.008	1.000	-0.022	-0.135	0.016	-0.060	-0.036	0.055
longitude	-0.005	-0.022	1.000	0.179	0.043	-0.152	0.039	-0.175
reflectivity	0.363	-0.135	0.179	1.000	0.134	-0.207	0.775	0.013
spectrum_width	0.048	0.016	0.043	0.134	1.000	-0.098	0.137	0.027
time	0.456	-0.060	-0.152	-0.207	-0.098	1.000	-0.166	0.015
total_power	-0.160	-0.036	0.039	0.775	0.137	-0.166	1.000	0.027
velocity	-0.041	0.055	-0.175	0.013	0.027	0.015	0.027	1.000

Count
Matrix

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

First rows
Last rows

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time
0	10.659610	106.728325	10.0	3.75	23.07	NaN	NaN	15.0
1	10.665006	106.728350	15.0	17.97	39.31	NaN	NaN	15.0
2	10.670401	106.728360	21.0	11.79	31.69	NaN	NaN	15.0
3	10.675796	106.728380	27.0	11.75	23.95	NaN	NaN	15.0
4	10.681192	106.728390	34.0	8.29	30.16	NaN	NaN	15.0
5	10.686587	106.728420	39.0	0.98	25.60	NaN	NaN	15.0
6	10.691983	106.728430	46.0	NaN	34.03	NaN	1.48	15.0
7	10.697379	106.728450	52.0	NaN	44.41	NaN	0.94	15.0
8	10.702774	106.728470	58.0	NaN	56.36	NaN	NaN	15.0
9	10.708171	106.728490	64.0	NaN	52.54	NaN	NaN	15.0

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time
702990	13.290660	106.68518	24244.0	NaN	NaN	NaN	NaN	133.0
702991	13.296014	106.68509	24304.0	NaN	NaN	NaN	NaN	133.0
702992	13.301366	106.68500	24364.0	NaN	NaN	NaN	NaN	133.0
702993	13.306719	106.68491	24424.0	NaN	NaN	NaN	NaN	133.0
702994	13.312071	106.68482	24483.0	NaN	NaN	NaN	NaN	133.0
702995	13.317425	106.68474	24543.0	NaN	NaN	NaN	NaN	133.0
702996	13.322777	106.68465	24604.0	NaN	NaN	NaN	NaN	133.0
702997	13.328129	106.68456	24663.0	NaN	NaN	NaN	NaN	133.0
702998	13.333482	106.68448	24724.0	NaN	NaN	NaN	NaN	133.0
702999	13.338835	106.68439	24784.0	NaN	NaN	NaN	NaN	133.0

Most frequently occurring

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time	# duplicates
14	10.65961	106.728325	10.0	NaN	23.13	NaN	NaN	160.0	4
4	10.65961	106.728325	10.0	NaN	22.96	NaN	NaN	49.0	3
8	10.65961	106.728325	10.0	NaN	23.02	NaN	NaN	8.0	3
16	10.65961	106.728325	10.0	NaN	23.14	NaN	NaN	167.0	3
25	10.65961	106.728325	10.0	NaN	23.18	NaN	NaN	163.0	3
27	10.65961	106.728325	10.0	NaN	23.19	NaN	NaN	157.0	3
28	10.65961	106.728325	10.0	NaN	23.19	NaN	NaN	164.0	3
31	10.65961	106.728325	10.0	NaN	23.24	NaN	NaN	66.0	3
39	10.65961	106.728325	10.0	NaN	23.32	NaN	NaN	116.0	3
44	10.65961	106.728325	10.0	NaN	23.33	NaN	NaN	125.0	3

Overview

Variables

Interactions

Correlations

Missing values

Sample

Duplicate rows

Most frequently occurring